Faithful Recovery of Vector Valued Functions from Incomplete Data Recolorization and Art Restoration

On March 11, 1944, the famous Eremitani Church in Padua (Italy) was destroyed in an Allied bombing along with the inestimable frescoes by Andrea Mantegna et al. contained in the Ovetari Chapel. In the last 60 years, several attempts have been made to restore the fresco fragments by traditional methods, but without much success. We have developed an efficient pattern recognition algorithm to map the original position and orientation of the fragments, based on comparisons with an old gray level image of the fresco prior to the damage. This innovative technique allowed for the partial reconstruction of the frescoes. Unfortunately, the surface covered by the fragments is only 77 m, while the original area was of several hundreds. This means that we can reconstruct only a fraction (less than 8%) of this inestimable artwork. In particular the original color of the blanks is not known. This begs the question of whether it is possible to estimate mathematically the original colors of the frescoes by making use of the potential information given by the available fragments and the gray level of the pictures taken before the damage. Can one estimate how faithful such restoration is? In this paper we retrace the development of the recovery of the frescoes as an inspiring and challenging real-life problem for the development of new mathematical methods. We introduce two models for the recovery of vector valued functions from incomplete data, with applications to the fresco recolorization problem. The models are based on the minimization of a functional which is formed by the discrepancy with respect to the data and additional regularization constraints. The latter refer to joint sparsity measures with respect to frame expansions for the first functional and functional total variation for the second. We establish the relations between these two models. As a byproduct we develop the basis of a theory of fidelity in color recovery, which is a crucial issue in art restoration and compression.